Intermediate or advanced level AWK

It is data processing language.
Everything is broken down into two parts: Patterns and Actions

Patterns are like regex patterns, searching for some text.

Actions describe what you would like to do with the searched patterns. Maybe you want to print the line containing patterns, or do something else (changing first letter of each word, maybe you want to reverse the word). Anything we do to the data, we call that Action

We will be following the gawk i.e. gnu-AWK, https://www.gnu.org/software/gawk/manual/gawk.html

Such commands are explained in this link

General awk command is awk 'pattern {action}' inputfile
If we wrote a program in awk named as program.awk then we can use it on some file.
awk -f program.awk inputfile

Printing hello world will require no input file: awk 'BEGIN {print "hello world"}' or awk 'BEGIN {print "hello world"}' inputfile both will print same, as BEGIN will execute the code before start of each file. So it does not need any file.
The following code awk 'BEGIN {print "hello world"} {print "hi there"}' inputfile will print : hello world hi there hi there hi there .. .. .. .. It will print "hi there" for each line of input file.
The following code awk 'BEGIN {print "hello world"} {print "hi there"} END{print"ending line"}' inputfile will print : hello world hi there hi there hi there .. .. .. .. It will print "hi there" for each line of input file. .. ending line it will print the 'ending line' after each line as printed
The following code awk BEGIN {print"this will print before first line of input file"} {print"here are some actions for each line of input file"} END{print"here I can add something as : program ended successfully (it require no input file just like begin )"}
If we don't use any pattern then action applies to each line: awk '{print"this action in curly braces will print for each line of input file"}' inputfile

we can write the program in awk:

		BEGIN
		{
		print "this will print even if we don't give any input file"
		}

		{print "This printing action will be applied on each line of inputfile"}
		END{print"This will be printed after each line of file has gone through some action above, i.e. it also does not require any input file "}

use awk -f program.awk inputfile to see the action. or we can make the program.awk an executable like that of a.out, or bash. by including awk shebang :

		#!/bin/awk
		BEGIN
		{
		print "this will print even if we don't give any input file"
		}

		{print "This printing action will be applied on each line of inputfile"}
		END{print"This will be printed after each line of file has gone through some action above, i.e. it also does not require any input file "}

In just one line expression we can use awk -v var='hi there' '{print var}' inputfile will print : hi there (for each line of input file)
awk -v var='10' '{print ""var""}' inputfile will print 10 for each line of input file.
awk -v var='10' '{print "var"}' inputfile will just print 'var' for each line of input file.

Using variables in awk scripts:

BEGIN {print "hi there"}
{var=10
print var
}
END {print "End of script"}

Search strings is regex statement, which allows us to select lines/words.

For example to find 'word' in input file and print each line ; we use awk '/word/ {print}' filename or we can simply use: awk '/word/' inputfile
we can pass multiple regex for searching; awk '/word/ {print} /e/ {print}' inputfile Above will print each line which have letter 'e' and word 'word'
For using multiple regex we can't use just patterns, we need at least one action for first pattern. awk '/word/ /word2/' inputfile won't work.

To print each line with length>50 (having more than 50 characters) we use awk 'length()>50 {print} inputfile'

	if (length($0)>max){ max= length($0)}
	else if (length($0)==max){print "Equal"}
	else {print "less"}
	END {print "maximum length is " max}

	 BEGIN {
	 for(i=0;i<9;i++)
		 {
		 print rand()
		 }
		 }


#!/bin/awk -f
#
# i will start from 1 by default
BEGIN {
while (i<10) {
		print i 
	i++}
	}

Let us say that we want to print out the total bytes added to files of directory in December month. Note that ls -l print out the

root@virat:~/phd_work/programs/awk$ ls -la
.rwxr-xr-x 156 root 24 Dec 13:22 conditionals.awk
.rwxr-xr-x  72 root 24 Dec 13:26 for_loop.awk
.rw-r--r-- 812 root 24 Dec 06:02 output_program1.txt
.rwxr-xr-x 190 root 24 Dec 06:12 program1.awk
.rw-r--r-- 517 root 24 Dec 06:01 readme.md
.rwxr-xr-x  93 root 24 Dec 12:48 variables.awk
.rwxr-xr-x  87 root 24 Dec 13:32 while_loop.awk

The second field contains number of bytes, 5th filed contains the month. We can write
ls -l | awk '/Dec/ {x+=$2} END {print "number of bytes added is " x} inputfile
or
ls -l | awk '$5=="Dec" {x+=$2} END {print "number of bytes added is " x} inputfile

NF : Number of fields (maximum field number) awk 'NF>0' inputfile will print each line which has at least one field(it will remove an empty line)
NR : Number of Records: or current line number $NR is not valid, it must contain a value NR=3, or NR=4. to print out the lines which are evenly numbered: awk 'NF%2==0' inputfile to print out total number of lines : awk 'END {print NR}' inputfile
To print nth line of file use: awk 'FNR==n {print; exit}' filename
You can use the awk command to count the number of lines in a file that contain a specific keyword, such as 'CHI_S = '. Here's an example: awk '/CHI_S = / { count++ } END { print count }' your_file.txt Replace your_file.txt with the actual path to your file.

Contents

Introduction

Pattern

Actions

Basics of AWK

variables in awk

Search Strings

If ..else..If Conditionals

For Loop

while loop

Examples

Special variables